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ABSTRACT 

This paper describes the field iapleaentation of a 
■ulti' level infor«ation feedback systea developed in the school 
District of Philadelphia, discusses soae of the probleas involved in 
its iaple»entation, and shares soae of the insights gained in the 
process. The prograa, which is designed to iaprove the reading skills 
of students, is oriented toward the instructional process, includes 
data related to pupil perforaance and pupil characteristics, and 
provides inforaation for adainistrators and teachers responsible for 
the education of over 100,000 of the school system's pupils, (RB) 
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An Information Support System for the Teacher as Instructional Manager 

James E. Scheib 



An all too unfamiliar statement about offices of research and evaluation 
is: 'Ve serve classroom teachers." Collecting and scorinq standardized tests 
administered by classroom teachers at the direcL.on of central administrators 
does not qualify for this motto. Seldom are research activities brought 
"down" to the scope of the classroom teacher. Unless you can affect classroom 
teaching, you do not influence education . except on paper. The great majority 
of classrooms retain "group" instruction despite many varied attempts at 
individualization of instruction. The reason for this is that most teachers 
are not able to generaie, analyze, and act on the tremendous amount of informa- 
tion needed to check, for each pupil, on the many specific objectives embodied 
in their curriculum. 

One of the greatest weaknesses in American education today Is the failure 
to help teachers do what we have known for decades must be done. Teachers do 
not need more techniques. Teachers need support In Implementing the proven 
methods so that the "minimum essential" skills are mastered by all, while 
more advanced students continue to move ahead. 

The most obvious way to reduce the wide variety of instructional levels 
for which the teacher must plan Is to group homogeneously. 

The remainder of the dilemma is left to the carefully balanced combination 
of group instruction and Individual guidance. Many approaches are possible 
here including the use of programmed materials or work books built around 
domain referenced measures. Most of the netv reading materials using these 
systems are expensive, consumable, and Involve a great deal of record keeping 
and testing. Here is where support can be given to the classroom teacher. 



Systematic instruction In the bcisic skills entails qood testinq tech- 
niques for placement, diagnosis, and mastery; the latter under the proper 
circumstances might be called grading and/or promotion. Test construct i on » 
production, scoring, and reporting are all too often left to the area of the 
teachers* informal measures. Although this Is often the only alternative 
for many areas of the curriculum it need not be the case for all, especially 
those minimum essential basic skills such as reading. With populations whose 
general language background Is seriously lacking, It cannot be assumed that 
if children do not get these basic skills In the early grades, that they will 

pick them up later on. 

Screening for children with serious language deficits should be system- 
atic and objective. You need instruments which can be used throughout large 
school systems without great expense. They must be developed Instruments 
whose criterion behavior Is Important In Itself In the sense of a criterion- 
referenced test. In addition, they should have at least "local" norms so 
that teachers know "how much Is enough for tiow." 

In an attempt to help managers of reading programs designed for deprived 
children In Philadelphia, Research Associates began to oevelop "mastery tests" 
in key areas of reading. The characteristics needed to both serve the teacher 
and generate "state-of-the-ski 11" Information for project managers were: 

1. The test should be able to be administered in a group sesslon- 

2. The test should yield Individualized results in a pattern 
easily examined like an Item grid. 

3. There should be a group summary by Item for group Instruction. 

if. Scoring should be objective and efficient with no more than a 
week's turnaround time. 

5. The test should have alternate test capabilities. 



The Sight a nd Sound Inventory which f am going to describe as an example 
of such an Instrumfnt meets these criteria. It is a p.tonics inventory that 
is aJmlnistered to a whole class at once: it is In multiple choice format, 
is machine scored, and the report is a computer printout. The report and the 
scoring program are those described here today by Frances Byers. 

In the 1970-1571 school year some portions of The Philadelphia School 
District mandated diagnostic testing using an Informal phonics Inventory 
developed by Morton Botel of the University of Pennsylvania. Teachers were 
to give the test, score it, and then build an Item grid showing which students 
answered each item correctly. The clerical task was unreasonable where 
cycling and team teaching resulted In one teacher constructing grids for 150 
students. Scoring was less than objective, and sun-narizing results for local 
norms was a bulky task. This test also was dependent upon writing ability. 

In the 1971-1972 school year the first version of the Sight and Sound 
Inventory was tried out. Form A consisted of 26 items: 18 Initial consonants 
and eight final consonants. Form B consisted of ^5 Items: 19 consonant blends, 
four digraphs, four final sounds of multiple letters, three rhyming sounds, 
nine vowel SDunds of single letters, and six vowel sounds when combined with 
other letters. The answer sheets were digitek forms with five choices per Item. 
The teacher read printed instructions Item by Item. The students were told 
what portion of words to attend to; the teacher would say each' word twice, and 
the students were to find the letters representing that sound among the choices 
for that item. On this version the students were to blacken the letter Itself. 
The choices had been printed in the locations of the response boxes. Alternate 
word lists were provided for later testings but the answer key and item tested 
remained the same. The test was given to a cross-section of elementary students 
In two oi- Philadelphia's eight districts. Item analyses were generated and most 
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items were revised. The general finding was that most look-alike distractors 
were not working weM , while the sound-atike distractors were. 

The second version was prepared over the summer with the cooperation of 
language-arts supervisors from the Curriculum Office. Form B was expanded to 
58 items and the answer sheets were again digltek forms, this time with the 
response boxes added. During the 1972-1973 school year the test was used 
extensively throughout the elementary grades in one quarter cf the city. 
Some of the analyses of one districts' scores are shown in Table 1. 

Table 1 



Fall 1972 Sight and Sound Results District 2 
The School District of Philadelphia 
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12. b 


26 
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20.8 


26 
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26 
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26 
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24.6 
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11 .0 


O.sk 
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Form: 


Level of 
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test 











X 


: mean 


of 


r total number of Items 


SD 


: standard deviation 


KR-20 


: Kuder Richardson formula 20 


Se 


: standard error of measurement 


N 


: number of students tested 


>v 


: better achievfnq students 


A A, 


: lower achieving students 
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The district is an inner city one where 25 of 27 elcfflentary schools 
presently qualify for Federal funds under Title I E.S.E.A. guidelines for 
educationally deprived children. Item fntercorrelatfons were found to be 
low. while internal consistency was high. Most students were apprpachinq 
mastery of Form A by third grade while fifth graders continued to show 
deficiencies on Form B. The results show fewer children were tested in 
grade one with Fo-m A and grade two with Form B. These were for the most 
part better than average for the district. Some data is included from 
low-achieving tenth graders to show that when used as a screening instrument 
with older students, results are similar to those in fourth and fifth grades. 

The third version of the test now in use is on a two-sided NCS (Nation?! 
Computer Systems) form. Minor changes include the use of nonsense syllables 
to control for spelling knowledge in some items and the addition of items 
with combinations of three letters. 

Recen. thinking about test validity is throwing more of the responsibility 
for this test characteristic toward the test user. This question is not: 
"Is this a valid measure?" (whatever that means), but rather: "Is this 
instrument useful for your purposes and how do you know that it is7" The 
Sight and Sound Inventory tests recognition of sounds as parts of words 
pronounced twice by the teacher. Responses are free from handwriting and 
encoding limitations. Guessing is a factor; therefore Interpretation by 
item cannot be literally Indicative of mastery. It asks; given the correct 
answer among five choices, is the student able to Identify correctly the 
letters which represent the portion of the word (beginning, middle, or end) 
to which his attention was to be directed by the Instructions? (terns missed 
probably need teaching or review. I terns marked correcly may have been 
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known or guessed* The pattern of fncorrtict re«^ponses should be the basis 
for planning remediation. Instructions include the warninq to teachers 
to substitute other words for those which colncidentally would be included 
in sight vocabulary lists being tauqht. Second qrade results for classes 
in four etementary schools representative of the district described above 
showed a correlation of +0.80 with the vocabulary subtest of the California 
Achievement Test - 1970 edition, Level I, Form A* Twenty of the 92 items 
on this subtest tested similar items as the Sfqht ^nd Sound Inventory. 

Most importantly^ the test has been used by teachers to plan remediation 
in basic reading skills with information which^ in most cases, would not 
have been available otherwise. When you provide data which teachers would like 
to have but can't generate > and you see them use it after it is provided, 
then you can say, '*We serve classroom teachers." 
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